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ABSTRACT 

The following topics of the annual report of the 
evaluation and assessment unit of the New Mexico State Department of 
Education are discussed: (1) standardized/norm- referenced testing 
program^ (2) objective-based testing program^ (3) assistance to the 
mutual action plan, 0) tectoiical assistance and consultative 
services to SDE staff and opefatiorial units^ (5) State evaluation 
advisory committee^ (6) opinion survey, (7) budget, (8) informatiqn 
dissemination^ (9) future planning, and (10) projected activities. 
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The major purposes of the Evaluation Unit 
efforts in 1972-73 were to continue field testing 
procedures and measures for the in5)lementation of 
objective^based evaluation at grades 6 and 9; to 
field test measures in the area of career education 
including vocational skills measures at grade 12; 
to continue coordination of ongoing testing programs 
at grades 1, 5, 8, and 12, and to improve the data 
utilization capabilities of educators at various 
levels as well as the general public. 



ANNUAL REPORT: 1972-73 FISCAL YEAR - EVALUATIQN' UNIT 
INSTRUCTIONAL SERVICES DIVISION 
NHV MEXICO STATE. DEPARTMENT OF EDUCATION 

I. imODUCTION : 

The passage of Senate Bill 1 of the New Mexico Legislature 
1969 provided a mandate for and gave in^etiis to the evaluation and 
assessment programs of the New Mexico public schools. 

The statute specified that the New Mexico State Board of 

Education would evaluate one- third (1/3) of the school districts in 

the State each year. Specific reference is contained m 77-2-2 of 

the New Mexico School Code - State Board duties. 

W. assess and evaluate for accreditation purposes 
at least one- third (1/3) of all public schools 
each year through visits by department personnel 
to investigate the adequacy of pupil activities, 
functional feasibility of public school and 
school district orgaiization, adequacy of staff 
preparation and other matters bearing upon the 
education of qualified students; 

The responsibility of inpiementing the provisions of S-^nate 
Bill 1 was originally placed with the research, development, and plan- 
ning division of the State Departnent of Education. Subsequent changes 
in the Department's organization and priorities have now placed this 
responsibility with the Evaluation and Assessment Unit of the Instruc- 
tional Services Division of the Department. 

Both the philosophy and the processes for evaluating schools 
have evolved from one of quantitative analysis of school facilities, 
equipment, staff, general program and curriculum offerings, to one 



which also. includes objective-based product evaluation of student pro- 
gress. This evolution has taken place in order to move toward the 
qualitative analysis of school programs and their effects on students. 
A process has been developed which provides for the periodic selection 
of cognitive skill objectives by committees composed of school staff 
members, teachers, parents, students, and local citizens. Committees 
in every school system of the state have now been involved in estab- 
lishing priorities on a comparative value scale relating to each ob- 
jective. Instruments or tests to measure progress in meeting these 
priority basic skill objectives have been developed. These are being 
field tested and analyzed. 

The Evaluation and Assessment Unit has coordinated these 
efforts with local school systems, community leaders. State Depart- 
ment of Education staff specialists and outside consultants. 

The Evaluation and Assessment Unit has also assumed respon- 
sibility fqr the statewide testing program which utilizes standardized 
tests. The Unit has been involved both directly and in a supportive 
role with the Mutual Action Plan (MAP) and has provided assistance 
to the statewide school accreditation effort. Responsibility for 
coordinating Senate Memorial 40 (Local Citizens Advisory and Eval- 
uation Committees) has recently been assumed by the Unit. A continuing 
activity of the Unit provides technical assistance for developing 
evaluation models and procedures for other units and divisions within 
the Department. In addition, the Evaluation and Assessment Unit 



supports the activities, efforts and objectives of the Instructional 
Services Division of th^ Department of Education as well as the over- 
all goals and objectives of the Department and the State Board of 
Education. 

II, STANDARDIZED/NORM-REFERENCED TESTING PROGRAM 

During October 1972, standardized tests were administered in 
each of the 88 school districts. 

The Otis Lennon Mental Ability Test (OLMAT) was administered 
to all first grade students. The Cbnprehensive Test of Basic Skills 
(CTBS) , and the Short Form Test of Academic Aptitude (SFTAA) were ad- 
ministered to all fifth and all eighth grade students. Additionally 
the American College Test (ACT) was administered to most college bound 
high school seniors and a few high school 11th graders. 

A. The Otis Lennon Mental Ability Test - Grade 1 

The instructions for this test, a pictorial 
, aptitude measure, are read to the student by the 
. teacher or other test administrator. The instrument 
is designed to measure the aptitude of children 
entering the first grade to perform the tasks expected 
of a student beginning school. 

A Spanish translation of the instructions for 
this test was prepared and used in a few pilot schools. 
The responses concerning the utility of reading the 
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test instructions in Spanish were mixed. Ad- 
ditional study is indicated to determine the 
validity of this process. The Otis Lennon 
Test was administered to 21,244 first grade 
students . 

Comprehensive Test of Basic Skills and the Short 
Form Test of Academic Aptitude - Grade 5 

These tests are administered together to 
show anticipated acliievement as well as the obtained 
score. Tlie California Test of Basic Skills measures 
student levels of achievement in reading, language, 
mathematics, and study skills. The Short Form Test 
of Academic Aptitude is designed to give verbal, non- 
verbal and total aptitude scores. Appropriate levels 
of the above tests^we^eL^administered to 23,960 fifth 
grade students in October of 1972. 

Comprehensive Test of Basic Skills and the Short Form 
Test of Academic Aptitude- Grade 8 

These tests are similar to the tests administered 
at grade five but. at a higher level appropriate to the 
grade. Tlie tests were administered to 23,550 eightli 
grade students during the month of October 1972. 

Specific in-depth information concerning the 
results of the standardized testing program is contained 
in the following three documents: 



1. New Mexico Statewide Testing Progran 1972-73 

2. Analysis of the Statewide Testing Program 
Results - 1972-73 

3. New Mexico School District Profile 1975 

These documents were prepared by the Evaluation 
and Assessment Unit personnel. Copies have been distri- 
buted to all school siqjerintendents and to the appropriate 
public officials^, Copies for review and analysis are 
available at the New Mexico State Department of Education. 

The procedure for administering the standardized tests consisted 
of assigning a test coordinator for each school di. ict so that test 
related conmunications could be channeled to and from the State Depart- 
ment of Education. Every effort was made to initiate two way communica- 
tion between State Department staff ajid local school authorities con- 
cerning the testing procedures. 

A. Pre- test workshops wer3 held in each district in 
September, an inprovement over the previous year 
when test coordinators and administrators were re- 
quired to attend regional workshops. Another in- 
novation was the videotape training packet which 
was used successfully in several districts. 

Teachers of the students tested administered the measures 
using specific instructions provided thm in the Manual for Adminis- 
tration. The ccMTipleted tests were then sent to the State Department 
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of Education for preliminary checking and then to the test company for 
scoring and for performing the designated statistical analyses. Statistical 
treatment for the tests consisted of processing answer sheets which were 
coded for each student to indicate the name, date of birth, sex, ethnic 
background, school district, size of district, and name of teacher. Score 
printouts were obtained for the school, the district and the State 
Department of Education. 

B. Five regional post-test workshops, attended by 

nearly 300 educators, were conducted by consultants 
from the test companies the fii Jt week in December. 
Benefiting from our first year of experience with 
this technique, the presentations were made more 
relevant to the needs of the local education agen- 
cies in the area of interpretation and use of test 
results. Many favorable comments resulted and 
requests for even more such assistance in the 
future have been received • 

Information derived from this program has 
been useful not only in helping LEAs improve ser- 
vices to their students but in interpreting edu- 
cational programs and needs to parents, the 
community, board members, and legislators. 
III. OBJECTIVE-BASED TESTING PROGRAM 

The objective -based testing program is a cooperative effort 
between the New Mexico State Department of Education and the local 



school districts of New Mexico to develop a process and design in- 
struments for measuring the progress of students in behavioral terms 
and to aid in deteimining the quality of the product of New Mexico 
education. 

Objective-based tests have been developed as instruments 
for measuring student knowledge, skills, and abilities. These in- 
struments were .developed by State Department of Education staff and 
outside consultant specialists in the construction of objective- 
based tests. An extensive "bank'Vor list of cognitive basic skill 
objectives prepared and submitted by local school district conmittees 
was used as the base from which the current objective bank has evolved. 
This bank of basic skill objectives includes goal statements in five 
areas, including conwunication skills, science, social studies, math, and 
the new area added this year, career education. From these objectives, 
local committees select priority objectives for their students as out- 
lined at the beginning of this document. Measures are constructed 
based xxpon objectives selected by New Mexico districts. For further 
infonnation on this process, note the Evaluation and Assessment Unit 
publication, "A Manual to Aid the Understanding and Implementation of 
Statewide Evaluation." 

Eighteen instruments were field tested during fiscal year 
1971-72, and revisions were made on the basis of the field test 
findings. Revised versions of the tests, which more accurately re- 
flected the needs and expectations of local school districts, were 



administered to all graduating seniors during the spring of 1973. 

A stratified sanple of xth and ninth grade students was 
also included in the objective-based testing program in 1972-73. Tlie 
purpose of the 6th and 9th grade sample was to provide for progress 
analysis at critical checlq)oints , to assist local school districts 
in the determination of program strengths and weaknesses, and to 
primarily field test the measures and procedures. 

An additional component of the objective -based testing pro- 
grain for this fiscal year consisted of the development of instruments 
to assess students' awareness of various aspects of career education. 
Tests were also developed and administered in several selected voca- 
tional skill areas, including Vocational Agriculture, Distributive 
Education, Office liducation. Auto Mechanics, and Home Economics. The 
vocational skill tests were administered to students conpleting a 
vocationally approved program. Both the career education measures 
(grades 6, 9, and 12), and the vocational skills measures were ad- 
ministered on a sample field test basis to assist in refining the 
measures and the procedures. 

Objective-based tests were administered to approximately 
18,500 high school seniors^ approximately 5,000 9th grade students, 
and approximately 3,200 6th grade students. 

The process involved in carrying out the objective -based 
testing program consisted of preparing the test items for each of 
tlie forty tests that were to be given. ITiese tests were printed 



in the Departjnent of Education Printshop* A conputer printout was pre- 
pared to provide the specifics of the test assignments by school, nuii]3)er 
of tests to be given, and the appropriate nun4)ers of testing groups 
within each school district* The printouts and labels for each 
packet of tests and test materials were supplied to the Evaluation 
and Assessment IMit and used as a basis for packaging materials for 
each of the almost 1,000 classrooms tested. 

The test materials were packaged at the Department of Edu- 
cation to be taken to the schools by the State Department of Education 
objective-based test coordiiiators. Thirty-three SDE professional 
staff mwrtbers served as test coordinators for the program. Each SDE 
test coordinator and each local test administratbr was provided with 
••A Manual for Teachers to Aid the Understanding and Implement at icn 
of Statewide Testing/' This fourteen-page booklet contained explicit 
instructions for administering the objective-based tests. Test coor- 
dinators were provided with training sessions for carrying out this 
special assignment* The role of the SM coordinator was to contact 
the local school superintendent who had already received information 
about the program following a statewide meeting for superintendents 
and local test coordinators. The local superintendent also received 
a printout showing the nuniber of students per test group, grade levels, 
and tests to be given. SIE coordina^rs then delivered the testing 
mterials to the local test coordinator, met with the teachers or 
staff members who were to administer tlie tests, and discussed the 
testing procedures. The SDE coordinators were at the school system 
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while the tests were administered in order to assist with problems 
-•as they occurred and to prepare the tests and answer sheets for 
immediate return to the State Department of Education. As the answer 
sheets were returned to Santa Fe, they were checked, labeled, and 
dispatched to the 'State Data Processing Center for processing. 

The objective-based testing program involved all public 
school systems in the state. The tests were administered over a 
two week period during the month of March 1973. Three to five tests 
weie administered to each student involved in the sampling. T^proxi- 
mate testing time for each student was two hours. For those students 
who took the vocational skills test, a slightly longer period of time" 
was required. 

The answer sheets for the objuctive-based testing program 
were designed by the Evaluation Unit so that they could be scored 
here in New Mexico by the State Data Processing Center. This re- 
suits in a significant reduction in turn around uime, or the time 
between when the answer sheets are received at the Department and 
the time when the printouts of results are received by the LEAs. 
Tapes of the results were sent to Educational Evaluation Associates 
for extensive analyses. These analyses will be studied by the Evaluation 
Unit Staff, SDE specialists, and local school authorities to determine 
the validity, utility and relevancy of the tests for measuring the 
product of New Mexico education and for the purpose of correcting 
deficiencies that are found in the measures. The results v/ill be 
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made available to local school districts to assist them in the assess- 
ment of their educational programs, and to help provide the schools 
with additional infoirnation for curriculum design and selection of 
program materials. The Instructional Division will utilize the data 
available for consideration through the Mutual Action Plan, 

The Evaluation Uhit and additional staff personnel are avail- 
able to assist local schools in the interpretation and use of the test 
results. The Iftiit staff has also worked with districts at other cri- 
tical points in the objective-based program. A siq)erintendents* work- 
shop was held in September, prior to the objective ranking process. 
Specific infoimation concerning the selection o£ priority objectives 
by the districts* objective ranking committees was provided. A super- 
intendents' meeting was also held in early March of this year prior 
to the administration of the objective-based tests. Specific adminis- 
tration procedures were discussed, along with general dialogue concern- 
ing the evaluation program as a whole. 

Information concerning the results of the objective-based 
testing program for the school year 1972-73 will be made available to 
legislative conmittees. State Department of Education staff, the State 
Board of Education, local school staffs and to the public in forms 
appropriate to the needs and interests of each group. Field test data 
derived from the 1972-73 objective-based tests will be utilized in the 
revision of the measures and procedures, and will not be available for 
actual program assessment. 
IV. ASSISTANCE TO THE MlfTUAL ACTION PLAN 

Staff members of the Evaluation and Assessment Unit also 
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served as representatives to ten local school districts. The M^^ 
effort involved a one week staff orientation session, many visitations 
to local school districts, analysis of accreditation reports, in-office 
visits with local school staffs, and other Department- LEA phone and 
postal communications • ~ 

^ ' In addition to the MAP assignments, the Evaluation and Assess- 
ment staff conducted meetings and workshops for all MAP personnel on the 
interpretation and utilization of test results* 

V. TCOjNICAL ASSISTANCE AND CONSULTATIVE SERVICES TO SDE STAFF AND 
OPERATIONAL UNITS : 

The staff of the Evaluation and Assessment Unit provided teclini- 
cal assistance to the Title I, ESEA program, the Title III, ESEA program. 
Vocational Education and Early Childliood Education. The assistance con- 
sisted of consultation and reccsninendations and design development for 
evaluation models to* meet the specific needs of these particular programs. 

VI. STATE PmUATION AD\aSORY CO^HTTEE 

The Evaluation and Assessment IMit has coordinated the activities 
of the State Evaluation Advisory Coimittee on testing and evaluation. The 
Committee, consisting of twenty members, has met eight times during the 
fiscal year and has provided a valuable linkage between the State Board 
of Education and local sdiool authorities* Suggestions and recommendations 
of the Statewide Advisory Committee have been incorporated inlo the Evalu- 
ation Unit's activities whenever possible and when deemed in the best 
interest of state goals and needs* 
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VII. SENATE MENPRIAL 40 - CITIZENS CatflTOES FOR EVALUATION OF SCHOOLS 

The Evaluation and Assessment Unit assumed responsibility for 
Senate Memorial 40 activities during this fiscal year. Senate Memorial 
40 charges the State Board of Education with the responsibility of 
establishing Citizens School Advisory Committees in all school systems that 
do not have evaluation procedures. The program has been operative 
for two years. .During fiscal year 1972-73, Advisory Committees have 
foniied in ten new school districts. These include Roswell, Carrizozo, 
Chama, Dexter, Jal^ Melrose, Pecos, Pojoaque, Truth or Consequences and 
Lordsburg. Plans are now being developed to form committees in ten 
additional^ school systems during the 1973-74 fiscal year. The Local 
Citizens Adxasory Comndttees submit a report of their findings and 
reconroendations to the Governor, the State Superintendent of Public 
Instruction, and to the Legislative School Study Comnittee for analysis 
and subsequent action. 

VIII. SURVEY OF OPINIONS AND REACTIONS TO PROPOSED PLAN FOR STATEWIDE 
EVALUATION 

During this fiscal year, a proposed plan for statewide evaluation 
was prepared and submitted to local school authorities and selected SDE 
staff members for their opinions, reactions and recommendations. An ac- 
companying questionnaire seeking reaction to specific elements of the 
plan elicited responses from 90 educators. Reactions were varied and 
provided valuable information for continuous study and analysis of the 
program. 

The results of this survey are being used along with other 
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factors as a basis for modification of the plan and for the develop- 
^nt of future activities that show promise of improving education 
through the assessment of education in New Mexico. 

IX. ZERO BASE BUDGET 

All objectives of the Evaluation and Assessment Unit's program 
were in direct concurrence and support of the overall goals and objectives 
of the State Department of Education and the State Board of Education. 
"iTie objectives and perfomance indicators as established in the Zero Base 
Budget were accomplished as well as additional tasks not predicted at the 
initial time of budget preparation. Some of these included preparation 
of materials for Department staff irembers on accountability, testing, and 
evaluation. The staff also prepared reports for legislators, made public 
appearances, cooperated with agencies interested in educational improvement, 
acted as liaison i/ith the State Library and other resource facilities, 
responded to nationwide inquiries relating to New Mexico's evaluation and 
assessment efforts, and provided technical assistance to non-public 
schools on school evaluation procedures. 

X. PUBLICATIONS AND DISSEMINATION 

The Evaluation and Assessment Unit prepared and disseminated 
the following documents during the fiscal year: Statewide Testing 
Proigram 1972-73 . i\nalysis of Statewide Testing Program Results 1972-75 , 
New Mexico School District Profile 1975 , Manual for Teachers to Aid 
the Iftiderstanding and Implementation of Statewide Evaluation , New Mexico's 
Statewide Evaluation System - An Introduction 1972-75 , Manual for MAP 
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Representatives to Aid the IMderstanding of Statewide Evaluation , Pro- 
posed Plan for Statewide Evaluation , A Look at the State: Results 
of Objective- Cased Testing , 

XI. REPORTS TO STATE BOARD OF EDUCATION 

Two progress reports were prepared and presented to the State 
Board of Education to provide information enabling the Board to make 
decisions, policies and regulations concerning the statewide evaluation 
and assessment programs. 

XII. FACTORS IN FUnJRE PLANNING 

A variety of factors will influence the future plans and pro- 
jections for the evaluation programs of the state. These factors include, but 
are not limited to. State Board of Education policies and regulations, 
budgetary limitations, input from the Statewide Evaluation Advisory 
Coiiinittee, results of analyses of previous activities, input from local 
school stperintendents, legislative considerations, considerations for 
the changing needs and expectations of students, teachers and the community, 
and developments, knowledge and understanding in the field of tests, 
measurement, assessment, and educational accountability. 

XIII. PROJECTED ACTIVITIES FOR 1973-74 

The projected activities for 1973-74 under consideration at 

this time are as follows: 

.Standardized testing at grades 1, 5 and 8 ^ 

.Objective-based groiq) assessment measures at grades 6, 9, and 12 

.Increased communication and participation with LEAs regarding 

Statewide Evaluation 
.Consideration and position on affective domain measurement 
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.Increased data availability for accreditation and program 

improvement purposes 
.Publishing useful documents within the area of evaluation, 

i.e., the Profile, Summaries of Test Results, guidelines 

for Better Utilization of Test Results, etc. 
.Providing data base and interim evaluation assistance for 

the Mutual Action Plan 
.Providing a statewide evaluation design for early childhood 

education programs 

9 
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